Automatic exposure (AE), automatic white balance (AWB), and automatic focus (AF), called 3A, are the three important techniques used in digital still camera (DSC) systems. AE controls the amount of light reaching a sensor and prevents overexposure and underexposure.
The objective of AE is to achieve a good balance of exposure in an image. AE contains a metering algorithm and an exposure control.
First, the metering algorithm estimates the amount of incident light on the sensor and calculates an appropriate exposure value (EV). Later in hardware control, the exposure control adjusts three related devices, an aperture diameter, a shutter speed, and a sensor sensitivity based on the exposure value.
When the incident light increases, the image is over-exposed and exposure control should decrease the aperture diameter, increase shutter speed, or decrease sensor sensitivity. Oppositely, when the incident light decreases, the image is underexposed and the exposure control increases aperture diameter, decreases shutter speed, or increases the sensor sensitivity. Automatic exposure control means estimating the amount of incident light and automatically adjusting the exposure control.
Automatic Multi-Pattern (AMP) is one of the AE metering methods of Nikon. AMP uses a lookup table and chooses a better AE metering algorithm based on several conditions.
The general AE metering algorithms have their advantages in different scenes, so we can not adapt a metering algorithm for all scenes. For example, center-weighted metering always is used when a subject is centrally located in center, spot metering is used in backlit scenes, and average metering is used in landscape scenes where an object is far away.
AMP solves this problem by deciding a suitable AE metering algorithm for each different scene. In AMP, they construct a lookup table based on a visual assessment of tens of thousands of pictures, computer analysis of the relationship between brightness patterns, optimum contrast, human evaluation, and so on. Therefore, the performance in most kinds of scenes would achieve a good result on average.
Another AE metering algorithm is disclosed by C. C. Yu in “Automatic Exposure with Fuzzy Control,” Master Thesis, Department of Computer Science and Information Engineering, National Taiwan University, 2004.
C. C. Yu describes two problems for AMP:
(1). A smooth transition is needed between each scene classification.
(2). A subject is not always in the central region.
In the AMP method, the subject is assumed to be in the center region. In photography, the subject may not always be located in the exact center, but the subject will be located in the center near the left, right, top, or bottom. Moreover, if the center region also includes other insignificant information, the subject information will be diluted. Subject information is very important in scene classification of AMP. If the subject is not located in the center, we will take the wrong pictures naturally. C. C. Yu's “Subject Growing” solves this problem.
In C. C. Yu's solution, it assumes the subject always crosses the center region, and it sets the gray region as the initial subject, shown in FIG. 1.
After initialization, C. C. Yu extends the subject with the new regions, adjoining gray region, by considering the following conditions of the new region and the subject region:
(1) The difference of luminance is less than 2.0 LV.
(2) The difference of hue is less than 30 degrees.
(3) The saturation of the new region is more than 0.5.
If the new region conforms to the three conditions, a program sets it as the subject region. In the end of subject growing, if the number of detected subject regions is less than 6 or more than 20, the subject is defined as the default setting as the gray regions in FIG. 2.
After the subject growing process, more precise subject regions are obtained and are used to calculate the contrast between the subject and the background for use with a multi-reference table later.
In real life, the light source is unstable and changing all the time. When the measured light intensity is just near the threshold between two weather or contrast conditions, AMP metering system will change the metering algorithms with the changing light. If the change is fast, we will see the display screen is flickering, and can feel uncomfortable.
Fuzzy control is the mechanism that simulates the undefined regions between the defined regions. It helps to smooth the transition between each condition. C. C. Yu uses fuzzy control to smoothly transition between different weather, contrast, and subject conditions.
Obviously, the subject growing process increases the probability to guess the correct subject location, and uses the subject information for more precise metering. However, the result of subject growing is not always correct. If the subject growing process finds a wrong subject region, the metering algorithm will use the wrong information to take flawed pictures.
Another issue from AE is Bracket Exposure. Bracketing is a technique used to take a series of images of the same scene at a variety of different exposures that “bracket” the metered exposure. In a general AE bracketing method, the camera will automatically take 3 or 5 frames with exposure settings between 0.3 and 2.0 EV differences. It is useful when users are not sure exactly how the image will turn out or are worried that the scene has a wide dynamic range.
When we take pictures using AE bracketing, we next select our favorite one from bracketing pictures. An optimal exposure selector (OES) objectively selects one favorite picture from AE bracketing pictures, which differ in exposure only. About the benefits, OES saves the effort that users need to select a favorite picture, and saves the memory space that bracketing pictures would occupy.
To select the favorite picture, researchers analyze the factors of person perceptions, such as brightness, contrast, and colorfulness.
In decision making, the prior art OES analyzes the factors that are related to the person perception and are influenced by exposure control. The following three factors are considered:
1. Intensity Mean
Intuitively, the light intensity factor is the most related to exposure control. In implementation, we take the mean of light intensity to judge the quality. The larger of the mean, the bright the picture is.
2. Standard Deviation and Entropy
Besides the light intensity factor, C. C. Yu also considers the contrast factor. Different exposure controls will result in the different contrasts, and people prefer the high-contrast picture. To analyze this factor, he calculates the standard deviation and entropy of a histogram. The larger the standard deviation, the wider the distribution of the histogram, thus the higher the contrast. The larger the entropy, the more uniform the histogram.
3. Colorfulness
People like colorful images. In application, C. C. Yu thinks colorfulness as the distance from the pixel to an origin point in CbCr coordinates. The smaller the distance, the grayer the pixel is.
However, the matching ratio (i.e., selecting the right picture) of the prior art OES is not good enough.